Analytic PCA construction for theoretical analysis of lighting variability, including attached shadows, in a single image of a convex Lambertian object

We analyze theoretically the subspace best approximating images of a convex Lambertian object under different distant illumination conditions. Since the lighting is an arbitrary function, the space of all possible images is formally infinite-dimensional. However, previous empirical work by Hallinan [7] and Epstein et al. [4] has shown that images of largely diffuse objects actually lie very close to a 5-dimensional subspace. In this paper, we analytically construct the principal component analysis for images of a convex Lambertian object, explicitly taking attached shadows into account, and find the principal eigenmodes and eigenvalues with respect to lighting variability. Our analysis makes use of an analytic formula for the irradiance in terms of spherical-harmonic coefficients of the illumination [1, 14], and shows, under appropriate assumptions, that the principal components or eigenvectors are identical to the spherical harmonic basis functions evaluated at the surface normal vectors. Our main contribution is in extending these results to the single-image case, showing how the principal eigenmodes and eigenvalues are affected when only a limited subset (the upper hemisphere) of normals is available, and the spherical harmonics are no longer orthonormal over the restricted domain. Our results are very close, both qualitatively and quantitatively, to previous empirical observations and represent the first valid theoretical explanation of these observations. Our analysis is also likely to be of interest in other areas of computer vision and image-based rendering. In particular, our results indicate that using complex illumination for photometric problems in computer vision is not significantly more difficult than using directional sources.